Skip to content

feat(connect): detect existing instance before install (disconnect + reinstall or cancel)#76

Merged
juan-malbeclabs merged 3 commits into
mainfrom
jo/connect-detect-existing-instance
Jul 10, 2026
Merged

feat(connect): detect existing instance before install (disconnect + reinstall or cancel)#76
juan-malbeclabs merged 3 commits into
mainfrom
jo/connect-detect-existing-instance

Conversation

@juan-malbeclabs

Copy link
Copy Markdown
Contributor

What

Re-running the curl … | bash one-liner over a host that already has a running edge-connect container would collide with the existing DoubleZero tunnel, container name, and host ports. The installers now detect an existing instance up front and handle it cleanly.

Added a new section (3b) to all three installers — scripts/connect.sh, connect-testnet.sh, connect-devnet.sh — placed right after the Docker daemon check and before any host prep, image pull, or docker run.

When a container named $DZ_NAME (default doublezero-edge-connect) already exists on the host, the installer now:

  1. Warns that an instance already exists and may be running.
  2. Prompts to reinstall or cancel.
  3. On reinstall: prints Uninstalling existing instance..., runs doublezero disconnect inside the old container (drops the tunnel cleanly to avoid collisions), then removes the container before continuing.
  4. On cancel: aborts and leaves the running instance untouched.

Non-interactive runs (DZ_ASSUME_YES=1) reinstall automatically.

Notes

  • doublezero disconnect is attempted only if the container is actually running; the docker rm -f covers stopped-but-present containers too. Both are best-effort (|| true) so a teardown hiccup doesn't abort the reinstall.
  • Kept identical across all three installers (they stay independent files, no shared lib).

Testing

  • bash -n clean on all three scripts.
  • Existing bats regression suite (tests/scripts/preflight_ws_port.bats) still passes.
  • Verified both new paths end-to-end via the stub harness: reinstall issues docker exec … doublezero disconnectdocker rm -fdocker run; declining aborts before any docker run.
  • CHANGELOG updated.

…reinstall or cancel

Re-running the one-liner over a live edge-connect container would collide with
the existing tunnel, container name, and host ports. The installers now detect a
container named $DZ_NAME before any host prep or docker run, warn that an
instance already exists, and prompt to reinstall or cancel. On reinstall they
print "Uninstalling existing instance...", run `doublezero disconnect` inside the
old container to drop the tunnel cleanly, then remove it. On cancel they abort
and leave the running instance untouched. Non-interactive runs (DZ_ASSUME_YES=1)
reinstall. Applied identically to all three installers.

@ben-dz ben-dz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The silenced disconnect failure orphans the exact state 3b exists to clean up. The pre-existing docker rm -f "$DZ_NAME" at connect.sh:478 already freed the name and ports before docker run — so the PR's only real new value is consent + graceful tunnel teardown. But the doublezero disconnect runs with output discarded and || true, and docker rm -f then SIGKILLs doublezerod. If disconnect fails, the GRE interface, routes, and on-chain user are orphaned in the host netns, the operator is told nothing, and the downstream fresh doublezero connect only warns on failure — so a collision reads as "install sort of worked." Fix: capture the disconnect status and warn loudly on failure.

The docker exec … doublezero disconnect is unbounded. No timeout, so a wedged doublezerod or a container in a Restarting loop hangs the installer forever — under curl | bash the user sees "Uninstalling existing instance..." and then nothing. The scripts bound other risky calls (curl -m 1 in cloud detection); this one isn't. His suggested fix resolves both majors at once: docker stop (bounded, 10s then SIGKILL) if doublezerod tears down on SIGTERM, or timeout 30 docker exec ….

Minors:

Headless behavior change is undocumented and mislabeled. Pre-PR, a no-TTY re-run silently reinstalled; now it aborts — breaking automation that doesn't set DZ_ASSUME_YES — and confirm collapses "user typed N" and "no TTY" into the same "Cancelled" message, as if a human declined.
The prompt isn't network-aware. All three installers share DZ_NAME=doublezero-edge-connect, so the testnet installer will detect a live mainnet container and, on y or DZ_ASSUME_YES=1, destroy the mainnet tunnel without ever saying which network the victim is on. Still better than the old silent rm -f, but the prompt should echo the container's image/env.
No bats coverage for any new branch despite the ready harness. Notably, the current docker stub returns an id for every ps, so it can't even distinguish the running-vs-stopped paths — he lists the three tests worth adding (running → disconnect-then-rm ordering, stopped → no exec, no-TTY → abort before docker run).

…ust headless detection

Address review of the existing-instance guard:

- Graceful, bounded teardown: replace the raw unbounded `docker exec … doublezero
  disconnect` with `docker stop`, so the container entrypoint's SIGTERM trap runs
  its own bounded disconnect (releasing the GRE tunnel/routes/on-chain session)
  before doublezerod is killed. The stop is timeout-guarded so a wedged or
  restarting container can't hang the installer forever.
- Warn loudly on orphaned state: after removal, verify the doublezero1 tunnel
  interface is gone from the host netns (the container is --network host) and warn
  if it lingers, and warn if the stop timed out/errored before the forced removal.
- Preserve headless automation: a no-TTY run with DZ_ASSUME_YES unset now
  reinstalls (with a notice) instead of aborting, matching pre-guard behaviour;
  an interactive decline still aborts, with distinct messaging.
- Robust TTY detection: probe an actual /dev/tty open rather than trusting `-r`
  (a readable inode still fails to open with no controlling terminal), which also
  sidesteps confirm()'s tty read in that case.
- Network-aware prompt: name the existing instance's env/image (all three
  installers share DZ_NAME, so the testnet installer can find a live mainnet one).
- Add tests/scripts/reinstall_existing.bats: running (stop-before-rm ordering),
  stopped (no stop), no-instance (no-op), and headless (reinstall, not abort).
@juan-malbeclabs

Copy link
Copy Markdown
Contributor Author

Thanks @ben-dz — all five addressed in the latest push.

Major 1 — silent disconnect failure orphans host-netns state. Reworked to tear the old instance down with docker stop instead of docker exec … disconnect. That drives the container entrypoint's existing SIGTERM trap (docker-entrypoint.sh), which runs its own bounded doublezero disconnect (only when a tunnel is up) before doublezerod is killed — so removal no longer SIGKILLs a still-connected daemon. And it warns loudly on failure: since the container is --network host, a failed disconnect leaves doublezero1 in the host netns, so after removal the installer checks for that interface and warns that the on-chain session/routes may be orphaned. A stop that times out/errors before the forced rm also warns.

Major 2 — unbounded exec can hang forever. docker stop is bounded by the container's --stop-timeout (60s) and additionally wrapped in timeout 90, so a wedged doublezerod or a container in a restart loop can't hang the installer.

Minor 1 — headless regression + collapsed message. A no-TTY run with DZ_ASSUME_YES unset now reinstalls (with an explicit notice) instead of aborting, matching the pre-guard silent-reinstall behaviour so automation isn't broken. A genuine interactive decline still aborts, with its own distinct message. While wiring this I found [ -r "$TTY" ] is unreliable — a readable /dev/tty inode still fails to open with no controlling terminal (cron/systemd) — so detection now probes an actual /dev/tty open, which also sidesteps confirm()'s tty read in that case.

Minor 2 — network-unaware prompt. The warning now names the existing instance's env/image (via docker inspect), so the operator sees which network's container they're about to destroy.

Minor 3 — no bats coverage. Added tests/scripts/reinstall_existing.bats with a smarter docker stub that distinguishes ps -a -q (exists) from ps -q (running) and flips to "up" after docker run: running → docker stop before docker rm before docker run; stopped → no stop; no instance → no-op; headless → reinstall (not abort). All green alongside the existing preflight suite.

@ben-dz

ben-dz commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

in the new 3b block, existing_env="$(docker inspect … 2>/dev/null | sed …)" is missing the || true that its sibling existing_img line has. Under set -euo pipefail, a failing command substitution in a plain assignment kills the shell — so a docker inspect failure (container removed between the docker ps -a detection and the inspect, or a daemon blip) silently aborts the whole installer before the "already exists" warning even prints.

… abort

The existing-instance env label built its value with a bare
`docker inspect … | sed | head -1` command substitution, missing the `|| true`
its DZ_IMAGE sibling has. Under `set -euo pipefail` a failing docker inspect
(container removed between the `ps -a` detection and the inspect, or a daemon
blip) — or `head` closing the pipe early on multi-line env — makes the pipeline
non-zero and aborts the whole installer before the "already exists" warning even
prints. Add `|| true` so the label stays best-effort.

Also harden the bats stub: `docker inspect` now emits realistic multi-line env +
an image (exercising the DZ_ENV extraction and the head-closes-pipe path) and can
be told to fail via STUB_INSPECT_FAIL. New test asserts a mid-teardown inspect
failure still detects, warns, and reinstalls; the running-instance test now
asserts the warning names the victim's env/image.
@juan-malbeclabs

Copy link
Copy Markdown
Contributor Author

Good catch @ben-dz — fixed in 0547a4c.

Added the missing || true to the existing_env line so it matches its existing_img sibling and stays best-effort:

existing_env="$($SUDO docker inspect -f '{{range .Config.Env}}{{println .}}{{end}}' "$DZ_NAME" 2>/dev/null | sed -n 's/^DZ_ENV=//p' | head -1 || true)"

Two failure shapes are now covered: a docker inspect that returns non-zero (container removed between the ps -a detection and the inspect, or a daemon blip), and — since it's a pipeline — head -1 closing the pipe early on multi-line env, which SIGPIPEs the upstream and trips pipefail even on success. Either way the label just goes empty; the "already exists" warning and the reinstall still proceed.

I also hardened the stub so this is actually regression-tested (it previously returned nothing for inspect, so it couldn't have caught this): docker inspect now emits realistic multi-line env + an image, and honours STUB_INSPECT_FAIL. New test — "docker inspect failing mid-teardown does not abort the installer" — asserts the installer still detects, warns, and reaches docker run; the running-instance test now also asserts the warning names the victim's env/image. All 7 green.

Note: on modern bash (5.x here) a bare x="$(false)" assignment doesn't actually trip errexit, but older bash (4.x on CentOS 7 / older Ubuntu servers — squarely in the curl|bash target set) does, and the head/SIGPIPE race bites everywhere — so the fix is warranted regardless.

@juan-malbeclabs
juan-malbeclabs enabled auto-merge (squash) July 10, 2026 18:08
@juan-malbeclabs
juan-malbeclabs merged commit 705015c into main Jul 10, 2026
7 checks passed
@juan-malbeclabs
juan-malbeclabs deleted the jo/connect-detect-existing-instance branch July 10, 2026 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants